Overcoming Catastrophic Forgetting by Neuron-Level Plasticity Control
نویسندگان
چکیده
منابع مشابه
Overcoming Catastrophic Forgetting by Incremental Moment Matching
Catastrophic forgetting is a problem of neural networks that loses the information of the first task after training the second task. Here, we propose a method, i.e. incremental moment matching (IMM), to resolve this problem. IMM incrementally matches the moment of the posterior distribution of the neural network which is trained on the first and the second task, respectively. To make the search...
متن کاملOvercoming catastrophic forgetting in neural networks
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they ...
متن کاملOvercoming catastrophic forgetting with hard attention to the task
Catastrophic forgetting occurs when a neural network loses the information learned in a previous task after training on subsequent tasks. This problem remains a hurdle for artificial intelligence systems with sequential learning capabilities. In this paper, we propose a task-based hard attention mechanism that preserves previous tasks’ information without affecting the current task’s learning. ...
متن کاملOvercoming Catastrophic Interference by Conceptors
Catastrophic interference has been a major roadblock in the research of continual learning. Here we propose a variant of the back-propagation algorithm, “conceptor-aided back-prop” (CAB), in which gradients are shielded by conceptors against degradation of previously learned tasks. Conceptors have their origin in reservoir computing, where they have been previously shown to overcome catastrophi...
متن کاملOvercoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation
Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previous tasks. This approach can be applied, for example, to prevent accident on autonomous vehicles by applying the knowledge learned on previous situations. In this paper we present a method to overcomes catastrophic forgetting that learns new tasks and preserves the perfor...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the AAAI Conference on Artificial Intelligence
سال: 2020
ISSN: 2374-3468,2159-5399
DOI: 10.1609/aaai.v34i04.5981